Hamiltonian Monte Carlo (HMC)
HMC Proposals
Problem: MH + GS work well, but proposals are inefficient, or difficult to get
Solution: use information about shape of target distribution to generate better proposals
HMC Proposals
\[
\frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}
\]
![]()
HMC Proposals
generate random momentum vector (a “flick” of the friction-less particle)
use a physics simulation that takes current position \(x_0\) and momentum \(w_0\) and generates a new state \(\left ( x_T, w_T \right)\)
accept with \(A = min \left (1, \frac{exp(-H(x_T, w_T))}{exp(-H(x_0, w_0))} \right )\)
Note: HMC tends to produce states with similar energy (energy should be totally conserved, theoretically), so acceptance rate is high
Detour: Energy
\[
H(q,p) = \underbrace{U(q)}_\text{potential energy} + \underbrace{K(p)}_\text{kinetic energy}
\]
HMC Proposals
Hyperparameters:
Leapfrog Step Size
Leapfrog Step #
HMC Acceptance
\[
A = min \left (1, \frac{exp(-H(x_T, w_T))}{exp(-H(x_0, w_0))} \right )
\]
when energy \(H(x,w)\) stays the same or decreases, acceptance prob is high
when energy \(H(x,w)\) increases, acceptance prob is low because the proposed sample is unlikely
HMC Acceptance
show curve and how discretized steps cause div when step size is too small
MCMC Diagnostics
Effective Sample Size (ESS)
Trace plots/\(\hat{R}\) (r-hat)
(HMC) Divergent Transitions
Trace Plots + \(\hat{R}\)
Question: have our chains converged to the target distribution?
Answer:
start chain from different values
compare samples from chains
should look like a fuzzy caterpillar (not kidding)
![]()
Trace Plots + \(\hat{R}\)
![]()
Trace Plots + \(\hat{R}\)
\(\hat{R}\) is a metric that summarizes similar information as a trace plot
\[
\hat{R} = \frac{\hat{V}}{W}
\]
where \(\hat{V}\) is the pooled variance of all the chains, and \(W\) is the within chain variance.
Rule of Thumb: R-hat < 1.1
Effective Sample Size
\[
N_{eff} = \frac{N}{1 + 2\sum_{t=1}^{\infty} \rho_t}
\]
How many independent random samples would we need to get the same amount of information as our MCMC samples give us
Measures efficiency of samples, but there’s nothing inherently wrong with low ESS.
Rule of Thumb: ESS > no. chains * 100
Divergent Transitions
In HMC, when we reject proposals when the estimated trajectory diverges from the true trajectory (measured by \(H(x_T, w_T) - H(x_0, w_0)\))
Lots of divergences often means biased samples because HMC cannot effectively explore one or more parts of the distribution.
